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ABSTRACT 

Data Envelopment Analysis (DEA) is an objective 
statistical technique for discovering the comparative ranking of 
organizations with respect to their efficiency. The technique, which 
assesses the relationship between resources (inputs) and outcomes 
(outputs) may also be used to discover what variables make 
organizations effective and efficient in order to improve them* This 
report explains the use of DEA to assess the effectiveness of schools 
and to bring about school improvement. The Organizational Health 
Inventory for Elementary Schools (Hoy, Tarter, and Kottkamp, 1991) 
was chosen as a framework for examining school effectiveness and 
efficiency* DEA has the advantage of allowing multiple sources of 
input and multiple outputs. The use of DEA is illustrated through a 
hypothetical study of elementary schools in an urban context that 
used the following school-level variables as inputs: student 
achievement in vocabulary, reading, and mathematics; attendance; 
stability; expenditures per student; number of teachers per 100 
students; and percentages of students for whom English is the home 
language. The linear programming of the DEA system allows the 
development of minimum and maximum efficiency scales. An appendix 
presents the DEA model for a specific case* (Contains 3 tables, 2 
figures, and 21 references.) (SLD) 
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Introduction 



Data Envelopment Analysis is an objective, statistical technique for discovering the comparative 
ranking of organizations with respect to their efficiency. DEA (as it is abbreviated) may also be used 
to discover what variables make organizations effective and efficient in order to improve them. 

For the purposes of this primer, effectiveness may be defined simply as the extent to which an 
organization attains its purposes or ends (also known as outcomes or outputs). Efficiency, on the other 
hand, is the degree to which an organization successfully uses its resources in combination so that it gets 
the most for its total set of resources. We’d like all organizations to be effective--to attain their ends to 
the fullest extent possible. Such a consideration requires that we also reckon with the scarcity of money, 
human time and energy, and other resources that are the means (or inputs) by which ends are attained. 

To illustrate, we might say that the Jones School and the Smith School are equally effective if 
they are identical in achievement test scores, graduation rates, parent satisfaction, and other outcomes. 
If the Jones School, however, has a smaller budget, students of lower socioeconomic status, and other 
limitations, we might characterize it as more efficient. Schools like Jones equal others in outcomes, but 
combine limited resources more effectively, and are therefore to be regarded as more efficient than— 
although equally effective as— Smith. 

Schools rarely have identical outcomes: They usually vary somewhat in the goals toward which 
they are working, their student composition, teaching staff, size, and other inputs. It is necessary, 
though, if policymakers aim to compare these schools, that some agreement be reached on a common set 
of objectives to be attained. Such an agreement will first identify the list of schooling outcomes to be 
used for comparison, and might specify that schools will be compared on their outcomes in mathematics 
and science and on average grade-to-grade promotion rate. Additionally, resources must be used well 
because they are limited and need to be combined in the best way to achieve the outcomes desired. 

Until recently, however, no technique existed to accomplish these two tasks simultaneously for 
organizations such as schools or hospitals. DEA was developed to enable the comparison of a set of 



schools or a set of hospitals or other organizations that produce many outcomes and do not operate in the 
private sector. It enables analysts and those responsible for schools and other such organizations to detect 
V what makes some work better than others in achieving the outcomes specified for them. 

A fundamental question is why school comparisons are made at all. Because of the National 
Commission on Excellence in Education’s A Nation at Risk (1983) and subsequent reform reports, there 
is great interest by policymakers and practicing educators in improving schools. One way to do so is to 
find those that are doing comparatively well and study their exemplary features. Information on rankings 
and their causes can be confidentially fed back to principals and others so they can take informed action 
for school improvement. It is with this purpose in mind that the present primer has been written. 

In addition, however, some citizens, especially in the business and policymaking communities, 
want to use rankings for an additional purpose--to evaluate schools, possibly with incentives for superior 
performance and sanctions for poor performance. Some states, such as Illinois and South Carolina, 
publish school rankings and plan such performance consequences. Unless school rankings are adjusted 
for student background, they can be misleading and unfair. That is an additional reason for DEA: It 
enables analysts to compare schools that are unlike in various ways. 

This primer is intended for nontechnical users— principals and other educational leaders. For 
those interested in gaining a technical understanding of DEA, an appendix is included. In addition, 
several references for further reading on the technique and on variables associated with school 
effectiveness and efficiency are included. 

Purpose 

This primer is designed to assist school principals and district-level administrators in 
understanding and using effectiveness and efficiency analysis as a diagnostic tool for making 
improvements in their schools. DEA is a mathematical programming technique used to assess the 
relationship between resources (inputs) and outcomes (outputs). Therefore, it is a helpful tool in 
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implementing effectiveness and efficiency analysis. It is aimed at serving the needs of managers and 
other administrators who need feedback on the efficiency with which their resources are being utilized. 

S - By improving efficiency, educators will be better equipped to raise school productivity, that is, to become 
more effective. In the field of education, efficiency is usually conceptualized with student achievement 
as an output and district- or school-level expenditures and other resources as inputs. DEA can provide 
information about the degree to which multiple inputs and outputs are being maximally utilized. In 
layman’s terms, DEA shows which resources are producing the greatest returns to investments. 

Concern for accountability in the field of education has been mounting for over a decade, 
especially in the wake of declining achievement scores and increasing expenditures. In 1987, over $160 
billion was spent on elementary and secondary education in the public sector (U. S. Government, 1985- 
1986). That was over 20% of all state and local expenditures. Pressures to improve effectiveness are 
felt at the district level by administrators and, in turn, by principals in the schools. DEA, as a tool for 
examining efficiency of resource allocation, can lay the groundwork for improving effectiveness. This 
is accomplished by providing school administrators with information about which resources produce the 
greatest returns to investment, given preexisting social and budgetary constraints. 

Literature Review 
Efficient and Effective Schools 

Educators, psychologists, and sociologists have contributed to an extensive literature based on 
effective schools. Edmonds (1979) was one of the first to systematically examine highly effective 
schools, particularly those serving the urban poor. Other contributions to the literature were made by 
Brookover (1979), Brookover and Lezotte (1977), and Rutter, Maughan, Mortimore, Ouston, and Smith 
(1979). Some of the more salient variables associated with effective schools were identified by Purkey 
and Smith (1983). These include the degree of curriculum articulation and organization; schoolwide staff 
development; parental involvement and support; schoolwide recognition of academic success; maximized 
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learning time; district support; clear goals and high expectations; orderly and disciplined school 
environment; and leadership of the principal, characterized by attention to the quality of instruction 
(Wang, Haertel, & Walberg, 1990). 

More recent studies of effective schools in large urban areas identified newer variables associated 
with learning. Research by Van De Grift (1990) identified close reporting of student progress; the 
principal’s selective influencing of teaching strategies; and expressing high expectations for pupils’ 
achievement as factors which influence learning in inner-city school interventions. 

A theoretical framework delineated by Wang et al. (1990) was employed in our research with 
effective and effective schools. A comprehensive set of variables related to learning which includes both 
cognitive and affective schooling outcomes was identified and analyzed through a "meta-review" and 
synthesis of research (Wang et al., 1990). This synthesis yielded 228 variables related to school learning, 
organized into 30 scales within six categories. The conceptual framework includes state and district 
variables, out-of-school contextual variables, school-level variables, student variables, program design 
variables, implementation, classroom instruction, and climate variables which were organized from the 
more distal to the more proximal factors. These are shown in Table I (Wang et al., 1990). An analysis 
of these outcomes confirmed that the more important variables associated with schooling outcomes are 
those that are more proximal to student learning as illustrated by quality and quantity of instruction, home 
environment, and student characteristics. 

More recently, Wang et al. (1993) expanded and updated their synthesis to provide a knowledge 
base for school learning. Their research represents a degree of agreement among experts about the 
factors important to school learning and the relationships among those factors. An extensive literature 
search comprised of reviews, syntheses and chapters in handbooks yielded over 11,000 statistical 
relationships. The results indicated that the proximal variables, including individual student 



characteristics and instructional and home environments, more strongly influenced learning than did the 

more distal variables, including demographic, organizational, or policy-level variables, 
v 

Organizational Health. Organizational Resilience. 

Organizational Learning 

Our research also draws upon the literature addressing organizational resilience. When applied 
to educational settings, resilience is based on the proposition that some schools are effective and efficient 
over time despite typical disabling urban conditions. These schools demonstrate evidence of powerful 
internal enabling conditions, such as high involvement of all the participants coupled with an 
organizational capacity for continuous learning. Our theoretical framework employs conceptions of 
organizational health and resilience to explain effectiveness and efficiency. These conceptions refer to 
the school’s social system and technical capacity as evidenced by measures of its effectiveness and 
efficient resource utilization (Anderson & Walberg, 1994). 

Organizational resilience requires evidence of superior performance. Such evidence can be linked 
to processes internal to the organization that result from design and redesign efforts to sustain 
performance. 

We reviewed the research on school health and chose the Organizational Health Inventory for 
Elementary Schools (Hoy, Tarter, & Kottkamp, 1991) for part of our work. The 37-item instrument is 
soundly anchored in theory and, although it has not to our knowledge been applied to urban schools, it 
appears appropriate for the tasks we wished to accomplish. Five subtests, representing the dimensions 
necessary for school health, comprise this inventory: (a) Institutional Integrity, (b) Collegial Leadership, 
(c) Resource Influence, (d) Academic Emphasis, and (e) Teacher Affiliation. These dimensions are 
associated with the four organizational functions—adaptation, goal achievement, integration, and latency- 
necessary for the growth and survival of organizations (Parsons, Bales, & Shils, 1953). Parsons and his 
colleagues defined these functions as follows: adaptation-the ability of an organization to respond to its 
environment; goal attainment— the ability to set goals and to achieve them; integration-the ability to 
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maintain cohesiveness among system elements; and latency—the ability to foster and sustain appropriate 
motivational driving value systems within the organization. Schools regarded as healthy "effectively meet 
the instrumental needs of adaptation and goal achievement as well as the expressive needs of social and 
normative integration” (Hoy et al., 1991, p. 68). 

This thinking about organizational health is also reflected in the efforts of researchers to 
understand high-involvement organizations (Lawler, 1986; Mohrman, Lawler, & Mohrman, 1992). 
High-involvement organizations empower all employees in the organizational hierarchy. In so doing, they 
foster organizational commitment, deep alignment, and high individual energy (Mohrman et al., 1992). 
The creation of high-involvement organizations requires self-conscious effort in design, information flow, 
and social support. Lawler (1986) indicates that such organizations show superior performance, and are 
flexible and adaptive in their problem solving. Leadership is critical in the development of high- 
involvement organizations. All of the characteristics found to be critical in defining healthy schools (Hoy 
et al., 1991) are present: visioning, clear focus, trust among the participants in the enterprise, 

collegiality, and shared leadership. Principal leadership, energized by a desire to achieve these ends, is 
likely to be found in such resilient schools. 

Organizational learning requires a clear sense of objectives, and continual monitoring of their 
accomplishment. Over time, such exercises build collective memory to be called upon in later problem- 
solving situations (Argyris & Schon, 1978). Resilient organizations detect changes in the environment 
requiring changes in the organization’s normal operating procedures. 

Hedberg’s (1981) work compares and contrasts individual learning and organizational learning. 
In particular, the difference between imitative learning and generative learning is drawn; and the 
interactive nature of learning is underscored. Effective learners and their environments change in the 
process of learning, and stability can often be a hindrance to learning. Organizational learning is a 



function of individual learning but is not the sum of such learning. Because organizations have memories 

that preserve the recollections of behavioral norms over time, organizations as well as individuals learn. 

Argyris and Schon (1978) show that real organizational learning takes place only when 

assumptions are reexamined for their fit with current reality (see also Morgan, 1990, on "learning to 

learn"). Argyris and Schon (1978) also distinguish single-loop and double-loop learning modalities: 

Single-loop learning is sufficient where error correction can proceed by changing organizational 
strategies and assumptions within a constant framework of norms of performance. It is concerned 
only with effectiveness. . . .[In] double-loop learning, there is [a] double feedback loop which 
connects the detection of error not only to strategies and assumptions for effective performance 
but to the very norms which define effective performance, (pp. 22-23) 

Thus our methodology can be seen to consist of a two-stage process: (a) identifying effective and 
efficient schools through effectiveness and efficiency analysis; and (b) testing for the presence of 
organizational health and a capacity for organizational learning postulated to be critical to the achievement 
of organizational resilience. 

The Case for Utilizing Effectiveness and Efficiency 
in School-Based Research 

Schools, in a manner typical of most organizations, particularly those in the public sector, 
produce multiple outputs using a range of inputs. Carrying out an assessment of the performance of such 
entities requires a measurement of the efficiency with which they use their resources in producing 
valuable outputs. For a variety of reasons, however, it has been difficult to assess such efficiency in 
public-sector organizations, schools being particularly difficult cases. Schools utilize material resources 
of various kinds, in conjunction with nonmaterial resources, to generate a range of outputs. Efficiency 
evaluation normally requires some estimate of the maximum possible output levels to be compared with 
actual performance levels. In the case of other organizations in the public sector, such evaluations have 
not easily been possible. Obstacles to these evaluations include the fact that outputs are often not 
measurable in ways which would allow for aggregation to a single measure. Furthermore, production 
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processes in fields such as education are usually not known in the same way that engineering descriptions 
of similar processes in industrial firms are known. 

Another limitation on efficiency assessment of public-sector organizations resides in the largely 
nonprice domain in which they operate, making evaluations of social usefulness of specific outputs 
difficult to achieve. In the case of schools, social usefulness can often be inferred from broad national 
and state policy, but it is still difficult to bring each organization’s production process, and therefore its 
outputs, in line with higher-level objectives identified with those policies. This difficulty is therefore 
inherent in the way such production units relate to higher-order governmental units in a policy sense. 
Schools could use national norms and state norms to guide their operations, and in recent years, much 
discussion has centered on this mechanism for improving schooling performance. Nonetheless, apart 
from knowing whether or not they meet such norms, typical analyses do not provide much guidance as 
to the reasons for either attainment or nonattainment of such objectives. 

One particular problem with such limitations is that production processes are either fixed and 
nonmanipulable or totally unknown. Schooling production processes evidence both of these difficulties. 
In the first place, some important elements of the school process are fixed either by custom or by some 
general understanding of what would be minimally necessary. The case of class size is a very obvious 
one. There are others, however. For example, some students do better with particular types of teaching 
resources and methods of instruction than others; and in general attention must be paid to these aspects 
of the learning environment. Given any set of resources which normally are not changeable in the short 
term, it is clearly important for such resources to be combined in the most efficient manner to produce 
the maximum in learning gains for students. 

Current methods for measuring efficiency are fraught with shortcomings. Ratio analysis, a fairly 
common methodology for assisting in the identification of efficient production units in various parts of 
the public and private sectors, might appear measurable, but does not provide unambiguous indicators 




8 



of efficiency. For example, the ratio of achievement scores to district spending may be misleading. In 
any situation, moreover, in which small inputs are utilized, an organization may score high on one ratio 
X , (having a high output-input ratio) but low on another. No theoretical argument supports placing higher 
significance on one ratio than another. Across organizations, these patterns of ratios are not likely to be 
constant, and so, in addition to the problem identified above, there is the added problem of determining 
which pattern is appropriate for comparison purposes. The analyst or the organization is simply presented 
with a measure set of ratios, with no way of making good sense of them. 

A second method often used to study efficiency in and among organizations is regression analysis. 
Here the organizations’ output level is represented as a function of-— that is, determined by--a set of 
inputs. This analysis generates an estimated relationship of inputs to outputs by isolating the magnitude 
of the effect on organization output by changing the inputs by one standard deviation unit. It also 
identifies the sign of the relationship, that is, whether the relationship between inputs and outputs is 
positive or negative. If this methodology is used for studying efficiency between organizations, those 
organizations that lie above the predicted (average) relationship of inputs to outputs are regarded as 
efficient, those below are inefficient. But this methodology also has difficulties. 

First, the measured relationship is only an average one, while what is being sought in 
consideration about efficiency, especially among similarly placed organizations, is a measure of best 
performance. Secondly, the shape of the production relationship must be specified in advance although 
that is precisely what is not known. Regression analysis can be therefore, quite inappropriate for studying 
questions of organizational efficiency. 

DEA solves these problems associated with ratio analysis and specification of the production 
function by allowing multiple outputs and inputs, as are typical with schools, to be considered 
simultaneously. Measures of efficiency derived are also sensitive to the sample of organizations-in our 
case, schools--being considered, and so organizations that are regarded as serving the same function (such 
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as schools in a public school district), may be compared with each other. The typical data envelopment 
analysis thus generates a set of measures of relative efficiency for the reference set of organizations. This 
information is of substantial value in determining which units are in need of assistance. Diagnosis of 
specific shortcomings characteristic of organizations which employ similar production models may be 
derived through comparisons allowed by DEA. 

Tools for Implementing Effectiveness and Efficiency Analyses 

In educational settings, the selection of input variables used in DEA often falls into two 
categories, discretionary and nondiscretionary. Discretionary variables may be influenced by policies set 
by the school or the district. Examples are teacher-student ratios, expenditures per student, teacher 
salaries, and time per school of math and language instruction. Nondiscretionary variables, chiefly 
student characteristics not controllable by educators, include the percentage of nonminority students 
(operationalized in these terms in order to provide a variable positively correlated with other efficient 
inputs), percentage of students not classified as poverty level, percent of students who are native English- 
language speakers, and stability, i.e., the percentage of students enrolled continuously throughout the 
year. 

As a means of facilitating discussion between school administrators, insight to questions like the 
following can be offered by DEA and its companion software (Bessent, Bessent, Elam, & Long, 1984): 
1) Given similar levels of discretionary resources and student characteristics, which schools produce 
greater levels of achievement? 2) Within the parameters imposed by available resources and student 
characteristics, what levels of effectiveness (achievement) can be expected? 3) What aspects of school 
operations can be improved? 4) What levels of optimum effectiveness can be obtained from best 
performance in the face of reductions in resource levels? 5) How well are schools meeting productivity 



Answers to these questions have facilitated discussions in a Texas school district regarding 
allocation of resources, setting of priorities, and enhancing communication with concerned public (Bessent 
> et al., 1984). Moreover, the employment of DEA in a time series analysis provides the basis for 
comparisons within a given set of schools at two or more observation points. The efficiency frontier will, 
in all likelihood, advance or retreat from one assessment to the next. By examining a school’s outputs 
relative to its inputs at the later point, changes in its position relative to the frontier can be determined. 
Performance that qualifies as efficient at one point may not be efficient subsequently. This would result 
from an increase in the magnitude of output-input ratios required for efficiency at the later assessment. 

District or school administrators can gain technical information about resource utilization after 
a single DEA application, with respect to a school’s efficiency relative to other schools. Multiple 
assessments can provide them with information pertinent to their own progress as well as their 
comparative performance. 

A more detailed, technical explanation of DEA is found in the appendix. 

An Urban Elementary School and its Comparison Set: 

Policy Implications 

To illustrate the multiple input/output situation in DEA, we focus on the hypothetical results of 
a scenario involving elementary schools in an urban context. The following school-level variables are 
inputs for the model: students’ achievement in vocabulary, reading, and math for the prior year; 
attendance; stability; expenditures per student; number of teachers per 100 students; and percentages of 
students for whom English is the native language spoken at home and who come from standard income 
homes. Current achievement in vocabulary, math, and reading are the outputs. 

DEA software produces several reports detailing measurement and efficiency listings for units 
(schools) under analysis. The Measurement of Local Frontier for the hypothetical case is presented in 



Entry-level outputs and inputs for the target school are listed under Unit Analyzed. The Linear 
Combination of Frontier Units are the corresponding levels of outputs and inputs for schools operating 
;• at peak efficiency (E = l. 00). Schools X, Y,-and Z, which lie on the frontier, operate at combined output 
levels of 3.300 for vocabulary, 4.032 for reading, and 4.000 for math. Their combined input levels 
include percent attendance of 72.85, student expenditures of $4019.51, and percent standard (native) 
English-speaking students of 54.66. These are some of the resource and outcome quantities associated 
with maximum efficiency. To the right of the frontier data are the lower bound inputs and outputs. 
These data indicate the quantities of these variables resulting from extending the frontier facet downward 
in order to envelop less-than-efficient schools. 

The multiplier, derived from the linear programming employed in DEA, produces increases or 
decreases in the inputs and outputs consistent with its sign. A summary of the multiplier’s effect on each 
variable is available in a report produced by PASS (Bessent & Bessent, 1988), the linear programming 
software which performs DEA. 

Complete listings of upper and lower bounds and technical efficiency for each school unit can be 
obtained from PASS, showing relative standing with respect to the frontier facet and its linear 
combination of inputs and outputs. The numbers of schools enveloped, efficient, and inefficient, along 
with minimum and maximum efficiency scales, are provided in the Efficiency Listing. 

The usefulness of DEA as a diagnostic tool derives in large part from the linkage it establishes 
between resources and outcomes. To probe this relationship, school administrators have used DEA to 
assess the efficacy of instructional expenditures by determining whether these investments are producing 
commensurate outcomes. As a result, policy decisions are better informed with the aid of formative 
productivity evaluation. Some of the functions of DEA output for the benefit of local school boards 
include the following (Bessent & Bessent, 1988): evaluation of yearly progress; site-based information 
for developing budget requests and curricular plans; superintendent audits of proposed plans to monitor 
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schools’ improvement goals, to assess whether resources are being maximally exploited, and to determine 
whether requests for additional resources are justified; and to assist in the management of valuable 
resources across schools in a district. 

To determine excess resource consumption, DEA produces "slack" values for each output and 
input value across all decision-making units (schools). Slack is represented by the distance from the 
frontier along the Y axis (Bessent, Bessent, Kennington, & Reagan, 1982). Essentially, slack measures 
inefficiency for each variable in the model. This is consumption which can be eliminated while 
maintaining the same amount of output. 

Detailed effects of slack values can be found in output augmentation and input conservation 
reports produced by PASS. Shortages for outputs and excesses for inputs are listed for each variable. 
Table III shows the output and input levels, adjusted output levels, and the slack measurements associated 
with each of these variables. Adding the shortages to their respective adjusted outputs and subtracting 
the excesses from their inputs shows the amount of inefficiency or slack. For example, adding the 
shortage to the adjusted reading outcome factor would put School A on the frontier level at 5.236. To 
reach efficiency for the input variable percent standard (native) English, we subtract 29.022 from 
100.000, yielding 70.978. 
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Appendix 



The DEA Model: Geometric Explanation of Efficient Resource Allocation 
(the two input, one output case) 

The geometric formulation in Figure 1 of the DEA model shows its properties and utility in the 
case of a single output and two inputs. Figure 1 is the geometric representation of a simple DEA 
problem, one with six schools, each producing a single output with two inputs, teaching and 
administrative resources. The axes define input/output space in normalized inputs (per unit of output). 
The managers of each school are presumed to be attempting to use resources as efficiently as possible, 
but may not know how other schools use scarce resources. On the basis of actual output levels and 
known resource availabilities, each school can be located in input/output space for the two inputs. 

Figure 1 presents the mathematical programming problem of minimizing input per unit of output. 
Maximizing output per unit of input is an equivalent way of stating the same decision objective by 
exploiting the duality of relationships of mathematical programming. 

Since the staff of each school tries to use its resources economically, those schools that have the 
lowest ratios of inputs to output are by definition the most efficient ones. In Figure 1, schools A, B, and 
C can be seen to be efficient. To illustrate, lines are drawn to join school A to school B and School B 
to school C. The line ABC represents the efficient set of schools and their efficient technologies for 
nonteaching producing the single output. Schools F, D, and E all achieve their respective output levels 
with greater amounts of resource use than is the case with schools A, B and C. In DEA terminology, 
schools A, B, C are efficient, and schools D, E and F are less than fully efficient. An important benefit 
of the DEA approach can now be demonstrated, since it is possible to calculate precisely the degree of 
inefficiency of schools F, D, and E. 

For example, D’s efficiency (or lack of it) can be measured by taking the ratio of OD’ to OD. 
Clearly this ratio is less than unity (which is what it would be if D were efficient. Since all units on the 
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line ABC are efficient, they form the reference units for the evaluation of inefficient or enveloped schools 
like D, which in this case is related to its reference school D’. 

With respect to school D, it is instructive to note the relationship of the line BC to its evaluation 
regarding efficiency. Clearly D uses the same ratio of normalized inputs to outputs as reference unit D’ 
which is on the line BC. Since all points on BC can be expressed as linear combinations of the 
coordinates of the points B and C, D’s efficiency can be evaluated using those coordinate values in 
conjunction with values to be derived from the solution of the implied mathematical programming 
problem for that unit D. A straightforward statement of the problem in algebraic terms is presented 
below. 

The line from the origin to school F does not cut a line segment but can be related naturally to 
both of the efficient line segments AB and BC. School E is clearly inefficient, and its level of 
inefficiency must be calculated by other means since line OE does not cut any part of ABC. Schools F 
and D are enveloped by Schools A, B and C while School E is not naturally enveloped by any of the 
efficient schools. 

The fact that unit E is not naturally enveloped suggests the need to construct facets with which 
it could be evaluated for efficiency purposes. Chames, Cooper, and Rhodes (1978) suggest the 
construction of a fictitious school G which differs from a rightward (horizontal) extension of C by an 
arbitrarily small amount, while Bessent, Bessent, Clark, and Elam (1988) construct their fictitious school 
from the extension of the facet BC to OE. The unit G is used by Bessent et al. (1988) to identify upper 
bounds of relative efficiency for E, whereas unit H is used to identify the greatest lower bound for such 
an evaluation. For units that are not naturally enveloped, then, upper and lower bounds are computed 
and become part of the output from the efficiency analysis. The case of the not naturally enveloped 
school turns out to be empirically more the rule than the exception, making this extension of the original 
DEA model an important one (Bessent et al., 1988). 
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It should be pointed out that the efficiency concept utilized here is that of technical efficiency. 
Schools that are evaluated using this methodology are evaluated only on the degree to which they achieve 
desirable weighted-input-to-weighted-output ratios in natural terms, not in value terms. For example, 
effects on test scores rather than their monetary benefits are estimated. 

Information Derivable from the Model in This Simple Case 

Returning to the example, each of the schools A, B, C, D, E, and F can be associated with a 
measure of technical efficiency depending on whether or not it is on the efficient frontier ABC. 
Efficiency levels are constrained to lie within the range from zero to unity, with schools on the frontier 
having levels of efficiency of unity, whereas those not on the frontier have efficiency levels of less than 
unity. 

It is important to note that the derived indices of efficiency are measured relative to the reference 
set. Therefore, a change in the reference set is likely to result in a change in the derived set of efficiency 
measures. In other words, the efficiency measures derived are interpretable only in the context of the 
reference set used to derive them. Accordingly, the derived measures of efficiency have greatest 
evaluative significance in analyzing comparable units. 

Extension of Efficiency Analysis to Multiple Output Case 

Figure 1 is the geometric representation of the analysis of organizational efficiency in the two- 
input, single-output case. It is possible to extend the single representation of efficiency determination to 
the multiple output case geometrically so as to retain the transparency of the analysis. One way to 
accomplish this is through the use of a familiar constraint from elementary economics called the 
production possibility curve. This is the approach we will follow here, since it is possible to demonstrate 
all of the characteristics of the data envelopment approach to efficiency analysis in the case of an 
organization with two outcome variables. Figure 2 shows this to be the case. 
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The curve ABFCDE typically represents the maximum combinations of the two outputs that can 
be achieved given the resources available to a particular organization. As resources are shifted for the 
production of one output as opposed to the other, the total output attainable for the organization as a 
whole will take the shape of the ABFCDE curve. It can be read differently, however, for the case of 
several organizations. Now the curve still represents the maximal amounts of output concerned, but the 
points ABCDE represent maximally efficient organizations. A point like G therefore, indicates an 
organization that is less than maximally efficient, although it uses resources in the same way as 
organizations B and C. 

Note that an extension of the ray from the origin through G onto the part of the curve ABCDE 
intersects the facet BC at point F. This is an important observation. Organization G, say school G, is 
less than fully efficient, and it is now possible to represent this shortfall of G from the maximally possible 
output combination represented by the fictitious school or organization F, by comparing the points G and 
F. GF/OF is the measure of the shortfall of G from the empirically maximal point F in terms of outputs 
one and two. Therefore OG/OF is a measure of G’s relative efficiency in comparison to F’s. This 
measure of relative efficiency can be shown to be equal to 1-GF/OF. If we were to assume, for 
simplicity’s sake, that GF = OG, then the ratio GF/OF is equal to 0.5, and OG/OF = 1 - 0.5 = 0.5. 
Thus G has a relative efficiency of 50% in relation to F, its fictional comparison unit on the maximal ray 
BC. School or organization G is now known to be only half as efficient as efficient unit F, which is a 
combination of B and C units. 

The analysis of Figure 2 corresponds exactly to the analysis of the single output case in being able 
to demonstrate efficient and less-than-efficient units, and by showing how one might measure the degree 
of shortfall from full maximal efficiency. What is assumed is that total resource availability remains 
constant as we move from unit A to unit E, the curve ABFCDE representing the empirically maximal 
amount of outputs shown to be derivable from the available levels of inputs. Data envelopment analysis 
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allows for the specific measurement of these levels of shortfall for all the outputs considered for each 
individual organization that is not on the boundary curve ABFCDE, and shows what the levels of output 
are for the organizations that are maximally efficient. Reading from Figure 2, if G were maximally 
efficient, it would be associated with OEj and OA 2 of outputs 1 and 2 and unit F, whereas the actual 
levels of the two outputs it has achieved are OE! and OAj, clearly less than OE^ and OA 2 . Actual data 
from a data envelopment analysis of Chicago public schools will show what these results might look like 
in an actual case. 
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Table I 




Illustrative Variable 



Category 1. State and District Variables: These variables are associated with state- 
and district-level school governance and administration. They include state curriculum 
and textbook policies, testing and graduation requirements, and teacher licensure, as 
well as specific provisions in teacher contracts, and some district-level administrative 
and fiscal variables. 

District-level Demographics and Marker School district size 

State-Level Policy Teacher licensure requirements 



Category 2. Out-of-School Contextual Variables: These variables are associated with 
the home and community contexts within which schools function. They include 
community demographics, peer culture, parental support and involvement, and amount 
of time students spend out of school on such activities as television viewing, leisure 
reading, and homework. 

Community Socioeconomic level of 



community 



Peer-Group 



Level of peers' academic 
aspirations 



Home Environment and Parental Support 



Parental involvement in assuring 
completion of homework 



Student Use of Out-of-School Time 



Student participation in clubs and 
extracurricular school activities. 



Table I (continued) 



Category/Subcategory* Illustrative_Variab^ 

Category 3. School-Level Variables: These variables are associated with school-level 
demographics, culture, climate, policies, and practices. They include demographics of 
the student body, whether the school is public or private, and levels of funding for 
specific categorical programs; school-level decision-making variables, and specific 
school-level policies and practices, including policies on parental involvement in the 
school. 



Demographic and Marker 
Teacher/Administrator Decision-Making 
School Culture 

(Ethos conducive to teaching and learning) 

School-Wide Policy and Organization 
Accessibility 



Parental Involvement Policy 



Size of school 

Principal actively concerned with 
instructional program 

School-side emphasis on and 
recognition of academic 
achievement 

Explicit school-wide discipline 
policy 

Accessibility of educational 
program (overcoming 

architectural, communication, and 
environmental barriers) 

Parental involvement in 
improvement and operation of 
instructional program. 
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Category/Subcatego ry* Illu strative Variable 

Category 4. Student Variables: These variables are associated with individual 

students, including demographics, academic history, and a variety of social, behavioral, 
motivational, cognitive, and affective characteristics. 



Demographic and Marker 
History of Educational Placement 
Social and Behavioral 
Motivational and Affective 

Cognitive 

Metacognitive 



Psychomotor 



Gender 

Prior grade retentions 

Positive, non disruptive behavior 

Attitude toward subject matter 
instructed 

Level of specific academic 
knowledge in subject area 
instructed 

Comprehension monitoring 

(planning: monitoring 

effectiveness of attempted 
actions, monitoring outcomes of 
actions; testing, revising, and 
evaluating learning strategies) 

Psychomotor skills specific to 
area instructed 
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Category /Subcategory* 


Illustrative Variable 


Category 5. Program Design Variables: These variables are associated with 

instruction as designed and with the physical arrangements for its delivery. They 
include the instructional strategies specified by the curriculum and characteristics of 
instructional materials. 


Demographic and Marker 


Size of instructional group (whole 
class, small group, one-on-one 
instruction) 


Curriculum and Instructional 


Alignment among goals, contents, 
instruction, assignments, and 
evaluation 


Curriculum Design 


Materials employ advance 

organizers 



Category 6. Implementation, Classroom Instruction, and Climate Variables: These 
variables are associated with the implementation of the curriculum and the 
instructional program. They include classroom routines and practices, characteristics 
of instruction as delivered, classroom management, monitoring of student progress, 
and quality and quantity of instruction provided, as well as student-teacher interactions 
and classroom climate. 



Classroom Implementation Support 


Establishing efficient classroom 
routines and communicating rules 
and procedures 


Classroom Instructional 


Use of clear and organized direct 
instruction 
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Table I (continued) 



Category/Subcategory* 


Illustrative Variable 


Quantity of Instruction 


Time on task (amount of time 
students are actively engaged in 
• learning) 


Classroom Assessment 


Use of assessment as a frequent, 
integral component of instruction 


Classroom Management 


Group alerting (teacher uses 
questioning/recitation strategies 
that maintain active participation 
by all students) 


Student and Teacher Interactions: Social 


Student responds positively to 
questions from other students and 
from teacher 


Student and Teacher Interactions: Academic 


Frequent calls for extended, 
substantive oral and written 
response (not one-word answers) 


Classroom climate 


Cohesiveness (members of class 
are friends sharing common 
interests and values and 

emphasizing cooperative goals) 



Subcategories are listed below the description of each broad category and are 
each illustrated with representative variables. For example, the first broad 
category includes two subcategories, "District-Level Demographics and 
Marker Variables" and "State-Level Policy Variables." 



Source: Wang, Haertel, & Walberg. (1990). What influences learning? A content 
analysis of review literature. Journal of Educational Research. 84 (i), 30-43. 
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Unit 


Combination of 


LOWER BOUND 




Analyzed 


Frontier Units 


Frontier Referents 


School 


A 




X 


Y 


Z 


Unit 


2480 




6360 


3170 


5420 


Year 


1989 




1989 


1989 


1989 


Multiplier 






-0.239 


-0.070 


-0.585 


Outputs 

VOCABULARY 


3.300 


3.300 


4.300 


4.700 


4.700 


READING 


3.800 


4.032 


4.500 


4.800 


3.700 


MATH 


4.000 


4.000 


5.000 


5.700 


3.800 


Inputs 

VOCABULARY 


3.400 


1.892 


4.100 


4.900 


4.000 


READING 


3.700 


2.849 


4.500 


4.800 


3200 


MATH 


3.700 


2.849 


4.500 


5.100 


3.400 


ATTENDANCE 


94.600 


72.853 


95.300 


94.000 


94.300 


STABILITY 


95.100 


73.238 


97.000 


85.300 


88.200 


BUDGET 


5219.346 


4019.511 


3405.293 


4269.785 


3515.260 


TEACHERS 


7.547 


5.812 


5.020 


6393 


4.762 


PSTANENG 


100.000 


54.662 


91.800 


90.900 


100.000 


PSTANINC 


66.900 


51.521 


83.000 


95.900 


69.400 



Variables listed above include: 



Outputs 

Vocabulary (1989) 
Reading (1989) 
Math (1989) 



Inputs 

Vocabulary (1988) 
Reading (1988) 
Math (1988) 



Inputs 

Attendance 

Stability 

Budget 

Number of teachers per 
100 students 
Percentage of native 
English speaking students 
Percentage of students 
from standard income homes 
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Table HI 



Efficiency: 

Output Adjustment 


77.012 
Factor 1.299 








Output 

Levels 


Adjusted 

Output 

Levels 


Shortage 


Levels if j! 

adjusted to | 

Frontier 1 


Outputs 










VOCABULARY 


3.300 


4.285 • 


0.000 


4.285 j 


READING 


3.800 


4.934 


0.302 


5.526 | 


MATH 


4.000 


5.194 


0.000 


5. 194 J 


Inputs 


Input levels 




Excess 


Levels if I 

adjusted to I 

Frontier 1 


ATTENDANCE 1 


94.600 




0.000 


94.600 | 


STABILITY 


95.100 




0.000 


95.100 | 


BUDGET 


5219.346 




0.000 


5219.346 


VOCABULARY 


3.400 




0.943 


2.457 


READING 


3.700 




0.000 


3.700 


MATH 


3.700 




0.000 


3.700 


TEACHERS 


7.547 




0.000 


7.547 


PSTANENG 


100.000 




29.022 


70.978 1 


PSTANINC 


66.900 




0.000 


66.900 1 





Variables listed above include: 



Outputs 

Vocabulary (1989) 
Reading (1989) 
Math (1989) 



Inputs 

Vocabulary (1988) 
Reading (1988) 
Math (1988) 



Inputs 

Attendance 

Stability 

Budget 

Number of teachers per 
100 students 
Percentage of native 
English speaking 
students 

Percentage of students 
from standard income homes 
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Input 2 (non-teaching) 
Output 





Rgure 1: Geometric Representation of DEA 
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l>UTPUT 2 
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Rgure 2: The Production Possibility Frontier 




OUTPUT 1 
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The National Center on Education in the Inner Cities 



The National Center on Education in the Inner Cities (CEIC) was established on November 1, 1990 by the Temple 
University Center for Research in Human Development and Education (CRHDE) in collaboration with the University of Illinois at 
Chicago and the University of Houston. CEIC is guided by a mission to conduct a program of research and development that 
seeks to improve the capacity for education in the inner cities. 

*s 

;■ A major premise of the work of CEIC is that the challenges facing today’s children, youth, and families stem from a 
variety of political and health pressures; their solutions are by nature complex and require long-term programs of study that apply 
knowledge and expertise from many disciplines and professions. While not forgetting for a moment the risks, complexity, and 
history of the urban plight, CEIC aims to build on the resilience and "positives" of inner-city life in a program of research and 
development that takes bold steps to address the question, "What conditions are required to cause massive improvements in the 
learning and achievement of children and youth in this nation's inner cities?" This question provides the framework for the 
intersection of various CEIC projects/studies into a coherent program of research and development. 

Grounded in theory, research, and practical know-how, the interdisciplinary teams of CEIC researchers engage in studies 
of exemplary practices as well as primary research that includes longitudinal studies and field-based experiments. CEIC is 
organized into four programs: three research and development programs and a program for dissemination and utilization. The 
first research and development program focuses on the family as an agent in the education process; the second concentrates on the 
school and factors that foster student resilience and learning success; the third addresses the community and its relevance to 
improving educational outcomes in inner cities. The focus of the dissemination and utilization program is not only to ensure that 
CEIC's findings are known, but also to create a crucible in which the Center’s work is shaped by feedback from the field to 
maximize its usefulness in promoting the educational success of inner-city children, youth, and families. 



CEIC Senior Associates 



Margaret C. Wang 
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Professor of Educational Psychology 
Temple University 
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Temple University 



Lascelles Anderson 
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Educational Research and 
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University of Illinois 
at Chicago 
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Associate Professor of 
Geography and Urban 
Studies and Director, 
Institute for Public 
Policy Studies 
Temple University 

William Boyd 

Professor of Education 
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University 
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Visiting Assistant 
Professor of 
Counseling Psychology 
Temple University 



H. Jerome Freiberg 
Professor of Curriculum 
and Instruction 
University of Houston 

Michael Goetz 
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of Economics 
Temple University 
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Temple University 

Howard Liddle 

Professor of Counseling 

Psychology 

Temple University 
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University of Minnesota 
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La Salle University 

William Stull 
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Temple University 
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Temple University 
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of Education 
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